Repository navigation
zephyr: ll-schedule: fix a race between task cancel and new schedule - #4898
Conversation
lyakh
left a comment
There was a problem hiding this comment.
Nice catch! What is happening here is that when a task is cancelled, the zephyr-ll scheduler only changes its status to cancelled and expects the next scheduler run to actually remove the task from the list. However, if instead the stream is unpaused before the next scheduler cycle gets a chance to run, then we come to a wrong conclusion, that the task is already scheduled, missing the fact that its status is "cancelled," so we don't schedule it again.
The fix is good, it can be done a bit differently but both versions are fine with me.
There was a problem hiding this comment.
alternatively we could put this test above line 316 inverted and avoid running the loop completely, but maybe this way it's a bit more obvious. Although this does come with a bit of performance penalty compared to the other version :-)
There was a problem hiding this comment.
@lyakh hmm, but we can't know for sure "task" is already on the "sch->tasks" list. So I'd keep it here.
I uploaded a new version now. I realized we also need to avoid re-adding the task to the "sch->tasks" lists.
There was a problem hiding this comment.
if it is not on the list, we anyway continue after the loop, right? Same as if it is on the list and cancelled. So, the list only makes sense for not cancelled tasks on it.
|
Hmm, Intel internal test plan #7544 is still showing error with (UPDATE) V1 patch. I have two local TGL machines and with both, without this patch, multiple-pause-resume fails with enough iterations. But in the test plan (TGL machine with nocodec tplg), there is still a failure. So it is possible more fixes are needed still. This fix seems to be still valid. |
|
@lyakh Hmm, let's hold off one this. I started to worry whether the cancelled entry in the task list is cleaned right after this patch. |
As task cancellation is an asynchronous operation, it is possible that the task is in SOF_TASK_STATE_CANCEL state when zephyr_ll_task_schedule() is called. With current code, the cancel takes precedence and task is not scheduled again. This shows up as errors in SOF audio test case for multiple-pause-resume where a set of pipelines is repeatedly paused and unpaused. If the race is hit, a request to pause is successful (and pipeline task is requested to be cancelled) but the subsequent release will not get schedueld at all leading to an IPC timeout. BugLink: thesofproject#4742 Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
0e102bc to
7a6cd1d
Compare
|
Pushed a new version. This addresses a problem of adding the same task multiple times in the task list. As all instances point to same task struct, they won't be cleaned up as expected. This should be harmless, but for some strange reason with this V2 patch, the multiple-pause-resume test passes also with TGL nocodec topology. |
Yes, the new version makes sense! Not sure to what part of your explanation "harmless" is referring, but adding the same element to a list multiple times without deleting it is definitely anything but harmless! |
|
CI showing some not tested DUTs and an unrelated issue on BYT. |
As task cancellation is an asynchronous operation, it is possible that
the task is in SOF_TASK_STATE_CANCEL state when zephyr_ll_task_schedule()
is called. With current code, the cancel takes precedence and task is
not scheduled again.
This shows up as errors in SOF audio test case for multiple-pause-resume
where a set of pipelines is repeatedly paused and unpaused. If the race
is hit, a request to pause is successful (and pipeline task is requested
to be cancelled) but the subsequent release will not get schedueld at all
leading to an IPC timeout.
BugLink: #4742
Signed-off-by: Kai Vehmanen kai.vehmanen@linux.intel.com